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ABSTRACT 


This Final Report for NASA/JSC Contract NAS 9-14915, "Independent Evalu- 
ation of Shuttle Flight Control System," presents a brief summary of the work 
performed on the Approach and Landing Test (ALT) Shuttle flight control system 
(FCS) and references a bibliography of the formal reports generated during the 
contract period which document the activities In det.11. In general, the acti- 
vities were grouped In four categories: (1) Independent evaluation of ALT FCS 
software design; (2) Independent evaluation of Software Development Laboratory 
ISOL) testing of ALT FCS software; (3) Independent evaluation of Shuttle 
Avionics Integration Laboratory (SAIL) ALT FCS testing; and (4) Independent 
evaluation of ALT flight test results. 
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1.0 INTRODUCTION 


This final report provides a brief summary of the analyses performed 
for the Avionics Systeins Engineering Division of NASA/JSC under Contract 
NAS 9-14915, "Independent Evaluation of the Shuttle Flight Control System," 
and references a bibliography of the formal reports, generated during the con- 
tract period, which document the activities In detail. The activities per- 
formed under this stud^ fall Into four general categories: (1) Independent 
evaluation of ALT FCS software design; (2) Independent evaluation of Software 
Development Laboratory (SDL) testing of the ALT FCS software; (3) Independent 
evaluation of Shuttle Avionics Integration Laboratory (SAIL) ALT FCS test- 
ing; and (4) Independent evaluation of ALT flight test results. Each of these 
activities Is discussed In the following sections. 
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2.0 DISCUSSION 


2.1 Indepen de nt Eva lu ation of ALT FCS Sof tware Des ign 

As a result of detailed Investigations of the HAL code for the FCS, TRW 
developed HAL signal flow diagrams (Reference 1-1) to provide roadmaps through 
the complex hierarchy of the FCS application software modules. These signal 
flow diagrams were manually produced and would be quite expensive to modify 
as the software changed. In response to their Inquiry, the Shuttle Program 
Assessment Office was Informed of this difficulty (Reference 1-2) and they 
have contracted with Intermetrics Inc. to develop an automatic signal flow 
generator. 

To effect a trace between the Functional Subsystem Software Requirements 
(FSSR) and the HAL code, TRW developed tables of Level C FCS FSSR signal traces 
to HAL code parameters and memory locations (References 1-3 through 1-6). 

These tables correlate the signals and constants In the ALT Level C FCS FSSRs 
to their corresponding HAL parameters and the memory locations of these HAL 
parameters. The Internal units of the HAL parameters were derived and re- 
corded In the tables. This tracing was performed for the aileron, elevator, 
body flap, and rudder/nosewheel control channels and proved valuable In mon- 
itoring FCS testing, defining and monitoring FCS troubleshooting procedures, 
and performing post- test analyses. 

Based on our Intimate knowledge of the FCS software, TRW performed 
several analyses to specify exactly how the software would perform In com- 
parison to the FSSR specification (References 1-7 through 1-10). These analyses 
Included a definition of the mechanization of the Program Test Input Program 
(stroking test); an evaluation of the automatic nosewheel steering gain; an 
evaluation of rudder transients during rollout; and the definition and pro- 
posed solution to GN&C sawtooth response due to navigation filter performance. 

In some cases, these analyses uncovered Initialization-load (I-Load) errors 
which would cause unexpected and undesired responses. In the case of the GN&C 
sawtooth response, a GN&C interaction problem was uncovered and a solution was 
recommended. The solution was not implemented during ALT due to schedule 
constraints. However, the solution is being considered for the OFT design. 
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This analysis pointed out the need for system-level analyses of the GN4C 
system to uncover Interaction problems. 

A major analysis Item began with detailed evaluation of the GN4C soft- 
ware and propagated to a general evaluation of the onboard flight software. 

The specific study was the evaluation of the use of the divide function In 
the GN4C software, from the standpoint of possible division by zero. TRW 
found that there were thirty-one divisions In the GN4C applications software 
which were not protected from division by zero (References 1-11 and 1-12). 

TRW determined that significant Impact could result from such errors, not 
only from the standpoint of the erroneous numerical result of this operation, 
but also from the standpoint of software operation. It was found that If a 
sufficient number of such errors occur during a cycle of a given software 
process, that process would stop (forced closed) for the remainder of the 
process. It was further determined by TRW that a number of other errors 
(overflows, unnormal Ized Input to floating-point divide, and overflow on 
conversion from floating-point to fixed-point) could result In similar prob- 
lems. These susceptibilities were reported at the ALT and OFT Orbiter Avionics 
Software Control Board (OASCB) meetings. The decision was made by the OASCB 
to accept the risks of these vulnerabilities for ALT but to add protection for 
the OFT software design. TRW worked with IBM and RI OFT design personnel 
(Reference 1-13) to familiarize them with the nature of the problem ->nd to 
establish a plan of action for OFT. TRW supported the developmet i • a 
Technical Directive to RI for specific OFT action required to provide pro- 
tection (Reference 1-14). 

It should be noted that the vehicle encountered division by zero and 
negative square roots during several of the ALT captive-active and free flights 
wh-’ch had not been predicted in SDL, ADL, or SAIL testing. The problems were 
not severe but the Incidents dramatically highlight the fact that actual flights 
subject the system to a much wider range of conditions than those simulated 
at the verification facilities. 

TRW also Investigated the API 01 General Purpose Computer (GPC) system 
software to establish how the system software performed the input/output (I/O) 



traffic through the GN&C appll'.atlon software (Reference 1-15). One specific 
PCS I/O transaction was examined to trace through the complex data transfers 
which are associated with the I/O traffic. The general observation from this 
examination Is that the complexity of, and limited visibility Into the system 
software appear to be Inappropriate for safety-critical, single-point failure 
software. 

2 . 2 Independent Evalu a tion of SDL Testing 

TRW participated In the IBM dry-run presentations prior to the ALT 
software Configuration Inspection (Cl). Module-level tests were reviewed 
and based on our Insight into the PCS software, TRW recommended some addi- 
tional testing (Reference 2-1). The total set of module-level tests and 
several GN&C system-level tests were evaluated at the ALT Cl. This Cl cov- 
ered the SDL testing of the software for the tailcone-off vehicle configur- 
ation. TRW submitted a number of Review Item Dispositions (RIDs) which 
addressed the completeness of the testing; correlation of test cases to the 
elements of software that were exercised; and the need for evaluation of sys- 
tetTi software performance under multiple GPC conditions (Reference 2-2). The 
reference also cites major observations from the Cl, Including: 

a. The objectives of the Cl were not clearly established; 

b. The volume of data and limited time available for Its digestion 
precluded a meaningful review; 

c. Much testing that was assumed to be In the purview of SDL testing 
was deferred to ADL and SAIL testing and the capability of these 
facilities are extremely limited; 

d. The nature of the data presented at the Cl did not allow a definitive 
assessment of the adequacy of the verification testing; and 

e. The fact that the SDL testing was limited to that defined in the 
Verification Test Specification (VTS) and the Verification Test 
Procedure (VTP) was not well known prior to the Cl or at the time 
of issuance of these documents. 

To provide a more comprehensive measure of expected GN&C performance 
for assessing SDL and other GN&C verification testino, TRW collected a set 
of GN&C performance requirements from various sources including the System 
Design Manuals (SDMs) and Certification Requirements (CRs) and provided this 
compilation to the technical community (Reference 2-3). These performance 
requirements were subsequently used by TRW In assessing the iCI test results 
as well as SAIL test results. 
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In response to the RID that we submitted at the Cl concerning correla- 
tion of testing to specific software elements, TRW generated examples of this 
correlation. Since *;he Shuttle system softwares design precludes a precise 
definition of all execution paths, "all" paths through the software cannot 
be verified. TRW reconnends that, at a minimum, the ^N^C software verification 
effort address exercising each executable statement In the 6N4C applications 
software at least once. TRW provided two techniques for systematically cor- 
relating the test cases to the software code to establish that this require- 
ment has been met (Reference 2-4). 

Preparatory to the aCI (tailcone-on vehicle configuration), TRW reviewed 
the results of software performance testing at the O^blter Aerodynamic Simu- 
lator (OAS). As descrloed In Reference 2-&, only limited Information could be 
extracted from these runs since the available data was restricted primarily 
to the downlink data and the runs were not flown by skilled crewmen. 

Also preparatory to the ACI, TRW performed detailed evaluations of 
eighteen GN&C system performance cases Implemented on the SDL. The GN&C 
performance requirements assimilated by TRW were applied and violations were 
noted. Response characteristics which did not violate any constraints but were 
unexpected were Identified. The results of this review and a summary of the 
TRW ACI participation are documented in Reference 2-6. The major facts from 
this review, the RIDs submitted by TRW, and evaluation of the testing philos- 
ophy include: 

a. Undesirable navigation/guidance/fl ight control transient interactions 
result from design deficiencies both in the requirements definition 
and in the software implementation; 

b. System performance requirements, as applied by TRW, were not accepted 
by the Cl Board as criteria for software verification; 

c. Comparison between the SDL and CSDL Statement Level Simulator (SLS) 
results yielded valuable qualitative Information on the software 
compatibility with requirements but is extremely expensive to im- 
plement; 

d. There is a need to establish realistic stress cases which establish 
the GN&C system performance envelope and it Is not economically 
feasible to Implement this work on the SDL, ADL, or SAIL; and 
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e. The CPU stress testinj was inadequate to establisn the Impact on 
GNiC performance of cycle overruns. 

IBM received an action from the aCI to perform additional CPU stress 
tests. TRW reviewed the results of these tests to establish the effect on 
GN&C performance. These reviews indicated that in one set of the stress 
cases, the Initial conditions varied sufficiently among the test cases to 
preclude a meaningful comparison among the runs. In a second set, the Ini- 
tial conditions were more similar but the results did not demonstrate the 
"graceful degradation" of GNtC performance that had been advertised. TRW's 
position Is that the Impact of cycle overruns on GN&C performance Is not well 
understood nor easily predictable. 

TRW continued to monitor the I -Load regression tests performed on the 
SDL by IBM to verify that the software would perform adequately for each ALT 
flight 1-Load. TRW established a technique for correlating the I-Load changes 
with the method for verification, which was adopted by IBM (Reference 2-7). 

Some results of TRW's review of the test results are documented In Reference 

2 - 8 . 

2. 3 Independent Evaluation of S A IL Testing 

TRW was a member of the ALT Certification Team created by the Avionics 
Systems Engineering Division. One of our major roles In this team was a review 
of SAIL test plans and test results and, prior *o formal testing, active 
participation in SAIL testing. 

TRW reviewed the SAIL Test and Checkout Procedures (TCPs) and provided 
corrections and additions for these detailed procedures (References 3-1 and 3-2). 
TRW participated In coordinated ADL/SAIL test planning for the ALT PCS (Ref- 
erence 3-3). 

Prior to formal certification testing, TRW provided active participation 
in SAIL testing and performed analyses of several Interim Discrepancy Reports 
(IDRs) and defined and supported a number of troubleshooting procedures to 
resolve IDRs. This participation included: 
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a. Nosewheel Steering Troubleshooting (Reference 3-4); 

b. Evaluation of SAIL roll and pitch steo responses (References 3-5); 

c. Evaluation of the Dedicated Display tests (Refe^-ence 3-6); 

d. Definition of IMU discretes associated with IMU Initialization 

(Reference 3-7); 

e. Evaluation of Incorrect mode response to PHC activity (Reference 
3-8); 

f. Analysis of SAIL navigation errors (Reference 3-9); and 

g. Evaluation of SAIL area navigation test (Reference 3-10). 

A number of the<^e analyses required evaluation of the math models within the 
SAIL simulation. 

TRW was a principal Investigator of the two SAIL certification runs for 
Free-Fllght 1 (Reference 3-11). We established the procedures and data re- 
quirements for this analysis which were used by the ALT Certification Team for 
analyses of subsequent SAIL rns. Additional evaluations of these two runs 
were documented In Reference 3-12. 

As a memuer of the ALT Certification Team, TRW performed detailed analyses 
nf certification runs for Free-Fllghts 2, 3, and 4. To assure timeliness, 
documentation of these analyses was made directly through the NASA/JSC re- 
porting system. Special reporting of problems generally was contained under 
the standard reporting with exceptions such as the reporting of non-statlonary 
aerosurface positions during a four second period when the primary GPCs were 
frozen and the Backup Flight Control System (BFCS) had not been engaged 
(Reference 3-13). 

In anticipation of OFT activities, TRW participated In several meetings 
In vhlch verification/certification testing at the SAIL and other facilities 
were discussed. These meetings were reported in References 3-14, 3-15, and 
3-16. 

2 . 4 Independent Evaluation of ALT Flight Test Results 

To provide a correlation of actual flight performance to the predicted 
performance, TRW supported the ALT Free-Flights 1 through 4 In the Mission 
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Evaluation Room and performed postflight analyses of specific anomalies 
or specific response characteristics. These analyses Included: 

a. Evaluation of eleven drift during CA-IA (Reference 4-1); 

b. Evaluation of fCS connand compatibility with rate gyro feedback 
(References 4-2); 

c. Analysis of rudder lag during CA-IA (Reference 4-3); 

d. Evaluation of erratic Horizontal Situation Indicator displays dur- 
ing CA-3 (Reference 4-4); 

e. Evaluation of FF-1 onboard state vectors (Reference 4-5); 

f. Observations of navigation state transients In FF-1 due to naviga- 
tion and MLS SOP computation frequency mismatch (Reference 4-6); 

g. Analysis of FCS tran.-pjrt delays in the CSS mode (Reference 4-7); 

h. Analysis of FF-3 onboard state vectors (Refe**cnce 4-8). 

The major observation on the ALT flight results is that the flights 
S'lbjected the onboard system to a much broader range of conditions than was 
used in the preflight verification/certification testing at the SDL, A^'l, 
and SAIL. This was particularly true of the navigation system. Indeed, there 
were a significant number of transients In the navigation state which had not 
been observed during preflight testing. Postflight '.nalyses indicated that, 
for most cases, the transients resulted from unique flight conditions. The 
concern is that these transients are generally undesirable and, in flight, 
proved to be the rule rather than the exception. 

2. 5 Work Plan Reporting 

The work plan and progress of the work were reported monthly in progress 
reports. These are documented in References 5-1 through 5-12. 
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